A Case For Lightweight Dynamic Event Based Monitoring and Management Support For Large Scale DataCenters

Future large scale systems, such as cloud datacenters, with increased core counts will soon result in infrastructures with millions of cores. This poses challenges for monitoring and management not yet met by existing experimental or commercial software systems. At the core of this challenge is to perform continuous and on-demand monitoring queries over distributed aggregated data resulting from distributed monitoring data streams. Of particular importance is the ability to quickly detect, correlate and react to system issues. Our research is developing monitoring and management methods and infrastructure that can scale and also exhibit small lag. Our approach is to use event based system design and distribute monitoring, select data aggregation and analysis and actuation across datacenter subsystems and machines. Finally, by embedding management into the underlying system and management infrastructure, based on modern support for virtualization, management is separated from applications, enabling its usage and change without affecting user codes. 1. PROBLEM DESCRIPTION Existing monitoring and management systems with single or multiple hierarchy for monitoring data aggregation and centralized analysis and coordination, are not designed to scale for future large scale systems. Some of the key challenges for monitoring and management systems are 1. Ability to proactively detect, correlate and react to system issues. 2. Ability to change part of monitoring data path due to changes in monitors, systems, subsystems or platforms. 3. Exhibit small lag in reacting to system issues caused due to external or internal factors affecting the systems. 4. Global and hierarchical correlated view of the information at an acceptable overhead and within an allowable precision. 5. Distributed coordination among various hierarchies of aggregated data, monitors and actuators across the environment. In particular, for large scale systems concerning our research such as future datacenters, key trends are shaping future technologies. First, there is an inexorable move toward many-core chips, which are increasingly composed of both general purpose cores and those specialized to certain tasks. Coupled with increased blade server densities and hardware disaggregation, these result in numbers of end systems and a degree of heterogeneity that makes it imperative to intimately integrate facilities for online and automated management into such systems. Second, increased demand for ever larger and more reactive datacenters, in part driven by cloud computing, will lead to scales of millions of cores, making it critically important for automated management methods to scale. It also implies the need for them to exhibit basic properties that include extensibility, the ability to interact with diverse management subsystems, and robustness. Third, virtualization is a becoming a necessary element of any study in systems management, in part because of its known benefits like server consolidation in datacenter environments. Thus two most important categories of system issues for monitoring and management systems focused in our research concerning future datacenter environment are 1. Dynamism With the growing complexity of various groups, applications and services in particular importance to the datacenter management is the ability to quickly detect and react to system issues, in order to mitigate and contain effects deleterious to application performance or datacenter health. The monitoring and management infrastructure should exhibit small lag under changes occuring due to dynamic and distributed virtualized datacenter environment. 2. Scalability The traditional approaches using centralized and reactive techniques for monitoring, data aggregation and analysis and actuation across datacenter subsystems and machines will not be able to scale to millions of core. Each of these components will have to be distributed and thus sheer scale introduces a problem of coordinated distributed decision across the datacenter environment. The monitoring and management mechanisms needs to scale in and scale out efficiently.